Test Manual

# Overview

Zadig manages the entire software development lifecycle, supporting almost all testing tools, services, and platform systems on the market, as well as multiple testing frameworks and different test types. Through powerful runtime environment governance and workflow capabilities, Zadig provides robust engineering support for testing teams.

This manual helps testing teams address current industry challenges by shifting testing services and capabilities into the entire lifecycle of development and operations teams. This enables early issue identification and empowers other roles to participate in quality assurance, avoiding additional costs from fixing such issues.

# Orchestrating Different Test Types

# Code scanning

# How to Configure

SonarQube example: Add code scanning, specify SonarQube scanning tool, configure codebase to scan, scan scripts, and enable quality gate inspection. See: Code Scan.

2. Configure code change triggers in code scan. See: Trigger configuration.

# How to Orchestrate

Edit workflow and add Code Scan task at specified stage (e.g., before building) to integrate code scanning into workflow.

# Unit Testing

# How to Configure

1. Add new test, configure basic information, code information, and test scripts, specify report directory in test report configuration. See: Testing.

2. Configure code change triggers in tests. See: Trigger configuration.

3. Configure IM notifications in test. See: Notifications Configuration.

# How to Orchestrate

Edit workflow and add Test task at specified stage (e.g., after deployment).

# Integration Testing

# How to Configure

Configuration process is similar to unit testing. See: Configure tests in Zadig

# How to Orchestrate

Edit workflow, add Test tasks, specify task type as Service Test. See: Workflow test configuration.

# System Testing

# How to Configure

Configuration process is similar to unit testing. See: Configure tests in Zadig

# How to Orchestrate

Edit workflow, add Test tasks, specify task type as Product Test. See: Workflow test configuration.

# Continuous Testing Run Scenarios

# Development Stage

Process: Code Implementation > Code Commit > Automatically Trigger Static Code Scan Quality Gate > Developers Receive Feedback > Targeted Improvements

After code development and PR submission, code scanning automatically triggers to effectively intercept code changes that don't pass quality gate. Scan results automatically comment on code changes, allowing developers to quickly obtain scan results and optimize code based on feedback. This helps avoid quality risks at source and ensures fail fast > feedback fast > fix fast.

### Testing Stage

Process: Static Scan (Quality Gate Enabled) > Build > Deploy > Automated Testing > IM Notification

Zadig can quickly set up independent development and testing environments. See: New Environment. By integrating test orchestration into the workflow, you can build automated test suites in both the development and testing environments, and incorporate testing capabilities into every aspect of the team's daily collaboration:

• In the self-testing phase, update the self-testing environment and execute automated tests.
• In the joint testing phase, multiple developers' changes can be deployed simultaneously for integration testing.
• During the test engineer's acceptance phase, test reports can be analyzed in Zadig, and the automation test suite can be continuously expanded based on coverage, ensuring that the automation test suite evolves alongside business functions, continuously providing value to the team and showcasing the test engineer's capabilities on the platform.

Additionally, the results of the workflow can be promptly notified to the IM group, allowing everyone in the team to stay informed about the automation execution and take responsibility for quality.

# Release Stage

Process: Release Quality Gate > Manual Review by Release Committee > Gradual Rollout > System Testing > IM Notification

After test acceptance, the release and deployment operation will be performed. Several configuration strategies are recommended:

1. Establish a release quality gate to automatically obtain quality results from security scans, unit tests, integration tests, etc., to determine whether the release is allowed. This serves as a checkpoint in the release process to ensure that the version has passed acceptance and meets quality requirements before going live.
2. Flexibly orchestrate release strategies such as MSE Grayscale, Blue-Green, Canary, Gradual Rollout, and Istio to ensure reliable releases. See: Release Strategies.
3. Add manual approval from the testing team during the release phase to ensure compliance in the business process.

# Continuous Testing Effectiveness Metrics

The effectiveness of continuous testing can be measured by the proportion of test cases written by different roles, the number of bugs found, the effectiveness of automated tests, and whether automated tests are run in CI/CD. Specific metrics include:

Measuring Factors	Metrics	Targets
Test Writers	Percentage of tests written by company developers, testers, and other members	The primary creators and maintainers of acceptance tests should be developers
Bugs Found at Different Stages	Proportion of bugs found over time	More bugs found during the testing phase
Time to Fix Acceptance Test Failures	Time required to fix acceptance test failures	Trend in fix time: Developers should be able to easily fix acceptance test failures
Effectiveness of Automated Testing	Reasons for test failures: Number of automated test failures due to actual defects and coding quality issues	Automated test failures always indicate actual defects in the product
Automated Testing in CI/CD Workflow	Check if all test suites run in each pipeline trigger (Yes/No)	Integration of automated testing: Automated testing should run in the main pipeline and main workflow

In addition, Zadig's performance insights can track all quality data throughout the software delivery lifecycle. You can customize XOps agile quality and efficiency dashboards to track metrics such as incident recovery time, defect escape rate, critical defect rate, and rework rate, quickly identify quality gaps, and use data to drive targeted and continuous improvement.